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DETAILED ACTION 

This office action is in response to the Amendment and Remarks filed 6/28/2010, 
in which claims 1 -21 were presented for examination. 

Response to Arguments 

Applicant's arguments with respect to claims 1-4 and 13-15 have been 
considered but are moot in view of the new ground(s) of rejection. Applicant contends 
the applied references do not describe or make obvious "a body suit" but merely 
describe where actuators are placed directly on the user or merely a body exoskeleton. 
This argument is moot in view of the newly applied reference Tremblay, see the 
rejection below. With regard to Applicant's assertion that the Rosenberg body 
exoskeleton is a sensor and does not "reflect back to the user through the exoskeleton" 
(Remarks page 3), the Examiner disagrees. "In addition to sending out a position vector, 
the FREFLEX [exoskeleton] controller receives as input a Cartesian force vector 
representing the magnitude and direction of the force to be applied to the user at the 
hand grip." (page 123, section 5.3.2.) With regard to Applicant's contention that while 
Rosenberg does describe computer-generated virtual fixtures, this is "merely a possible 
concept" (Remarks page 3), the Examiner disagrees. Further chapters, as referred to in 
the rejection, detail the testing of the virtual fixtures and thoroughly detail the toolbox 
software of virtual fixture elements available for the user to select and use. The software 
package to demonstrate the virtual fixtures is thoroughly discussed and presented in 
Chapter 7 of the thesis beginning on page 170. 



Application/Control Number: 1 0/734,61 6 Page 3 

Art Unit: 3661 

Remarks regarding the separately rejected dependent claims 5-12 and 16-21 
merely repeat the arguments as applied to claims 1 and 13. Please see the discussion 
above. 

Claim Rejections - 35 USC § 1 12 

The following is a quotation of the first paragraph of 35 U.S.C. 112: 

The specification shall contain a written description of the invention, and of the manner and process of 
making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the 
art to which it pertains, or with which it is most nearly connected, to make and use the same and shall 
set forth the best mode contemplated by the inventor of carrying out his invention. 

Claims 1 and 13 are rejected under 35 U.S.C. 112, first paragraph, as failing to 
comply with the written description requirement. The claim(s) contains subject matter 
which was not described in the specification in such a way as to reasonably convey to 
one skilled in the relevant art that the inventor(s), at the time the application was filed, 
had possession of the claimed invention. Claim 1 recites the amended limitation: "from 
the computer storage medium to feed the tactile signals...." The specific portion "to 
feed" is not described, defined, or suggested in the original disclosure. Claim 13 recites 
"and feeding" in line 8-9, which is not explicitly described to defined in the original 
disclosure. 

The amended portion has been examined as best understood in light of the 
specification. 

Claim Rejections - 35 USC § 103 

The text of those sections of Title 35, U.S. Code not included in this action can 
be found in a prior Office action. 
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Claims 1-4, and 13-15 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Simmons, US 6,741 ,911, in view of Mcintosh, US 5,103,404, in 
further view of Rosenberg, "Virtual fixtures': Perceptual overlays enhance operator 
performance in telepresence tasks" Stanford University, Stanford, CA, August 1994. 

(Claims 1 and 13) Simmons describes a virtual reality encounter system and 
method comprising: a humanoid robot having tactile sensors positioned along the 
exterior of the robot (column 8, lines 39-50), the sensors sending tactile signals to a 
communications network (column 7, lines 29-32, robot contains sensors of movement 
and pressure, column 10, lines 49-52); and a user body suit having tactile actuators 
(external actuator assemble and assistive equipment, column 6, lines 33-51 , column 8, 
lines 39-50), the actuators receiving the tactile signals from the corresponding tactile 
sensors on the robot from the communication network (column 7, lines 29-32), wherein 
the tactile sensors and the corresponding tactile actuators are calibrated in connection 
with variable sensitivities associated with different regions of the human (column 8, line 
62- column 9, line 4, column 1 3, lines 3-28). Further Mcintosh teaches that it was well 
known in the remote robotic control art to calibrate sensors to different levels of 
sensitivities to overcome the problem and allow, according to Mcintosh, individuals to 
vary the sensitivities of tactile feedback to optimize their own degree of sensitivity and 
control over the manipulator (column 1, lines 41 -50, column 9, lines 12-52). It would 
have been obvious to one of ordinary skill in the art at the time of the invention to 
calibrate the tactile actuators of the body suit, because as Mcintosh suggests this would 
allow the user to preset and vary the sensitivities of the suit actuators in individual body 
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locations to the user's preferred degree of sensitivity of how to feel the remote 
environment. This would as Simmons suggests, allow the user to feel pressure or force 
that is proportional to the robot at remote locations (column 40, lines 39 -45), allowing 
the user's hands to feel more delicate level of sensation or allowing the user to feel 
superhuman, by feeling only a little of a large force (column 40, lines 1 9-64). 

Simmons does not explicitly describe generating supplemental virtual tactile 
sensation signals from data retrieved from a computer storage medium to feed the 
tactile signals from the robot tactile sensors and the generated supplemental tactile 
sensation signals. However, Rosenberg teaches the use of perceptual virtual overlays 
in a telepresence system. Rosenberg defines perceptual overlay as "abstract sensory 
information overlaid on top of the reflected sensory feedback from a virtual, remote, or 
real environment... perceptual overlay may be composed sensory information ... 
including visual, auditory, haptic, and tactile sensations" (pg 1 1 , section 1 .8, lines 1- 
7). Further, "perceptual overlays are in fact virtual constructs, and not truly a part of the 
distal environment, perceptual overlays can be seen as embellishments of the user's 
perceptual reality" (pg 12, lines 1-2). The telepresence system includes a remotely 
controlled robot having force sensors (MERLIN robot arm, located in the "remote 
environment"); an exoskeleton having force actuators (FREFLEX upper body 
exoskeleton, located in the "operator space" . . . "where the human subject is interfaced 
to the system", figure 5.1 , pg 122 and the tested embodiment of a force reflecting 
joystick, pg 172), a gateway device (microprocessor located at the operator space 
connected to the user interface exoskeleton and/or joystick, pg 77 and tested as Intel- 
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486 computer, section 7.1 .1 , pg 172); inherently the microprocessor comprises a 
memory; a computer storage medium storing data for generating supplemental virtual 
tactile sensation signals (the microprocessor comprising a software package to be 
executed by the computer which to run on the microprocessor would inherently require 
it to be stored on a computer readable medium); the microprocessor configured to 
execute computer instructions stored in the memory (MERLIN VME processor, section 
5.4.4) configured to: retrieve data from the computer storage medium (MERLIN VME 
processor received input from the TASK Controller computer which fixture models 
should be generated; fixture models computed by the MERLIN VME, page 133, 5.4.4., 
lines 2-15); generating supplemental virtual tactile sensation signals from the data 
retrieved from the computer storage medium (virtual fixture models computed by 
MERLIN VME, fed to the FREFLEX VME controller, section 5.4.4, pg 133) to feed the 
tactile sensors and the generated supplemental tactile sensation signals to the tactile 
actuators to the user suit (Since "to feed" or "feeding" is not explicitly described in the 
original disclosure, in light of the specification the limitations have been interpreted as 
sending the signals. Section 5.4.4.2-5.4.4.5, the virtual fixture models are composed of 
mathematical models, which was used to determine the resultant force to be applied to 
the exoskeleton; the processor at the operator sight sends both virtual fixture signals 
and remote perceptual information from the remote site by overlaying the virtual fixtures 
directly on top of the perceptual information reflected from the remote environment, 
page 175, lines 1-5). 
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Rosenberg further teaches that the virtual fixtures may be implemented by the 
user by using a software package, Percept-Station, that includes a retrievable 
computer-program "toolbox" of virtual fixture models (abstract perceptual elements), that 
the user may overlay on the perceived remote environment by using associated 
keystrokes and menu options (pg 171, lines 14-33). The haptic sensations are 
produced by stored computational algorithms that are implemented by the user 
"choosing from among a menu of possible perceptual elements and then drawing a 
graphic representation of the percept at the desired location in the remote environment." 
(page 1 75, 7.1 .2.4, lines 1 -9.) It would have been obvious to a person of ordinary skill in 
the art at the time of the invention to combine the teachings of Rosenberg to the virtual 
reality system of Simmons in view of Mcintosh because as Rosenberg suggests "by 
overlaying particular configurations of perceptual information on top of the reflected 
sensory feedback from a telepresence worksite, operator performance can be greatly 
enhanced in a variety of manual tasks." (Pg 1 2, lines 8-10.) 

Rosenberg does not explicitly describe the operator's computer as a "gateway 
device". However, it was well known in the computer arts that a gateway can be 
embodied in a general computer to perform the protocol conversion or the functions that 
a separate gateway device (such as a router or modem) performs. It would have been 
very obvious to one of ordinary skill in the communication arts to utilize a general 
computer with an internal gateway device (internal router or modem) as the operator's 
computer which overlays the virtual fixtures, as this would have been merely a design 
choice of the network system, comprised of well known network components. 



Application/Control Number: 1 0/734,61 6 Page 8 

Art Unit: 3661 

Simmons describes a user assembly comprising actuators and sensors to "apply 
pressure to, mobbing, or any combination thereof" (column 42, lines 17-20) and monitor 
the bodies and limbs of the user using body worn sensors (column 7, lines 29-40). 
Rosenberg teaches an exoskeleton worn by the user (Figure 5.1 , page 122). Simmons 
and Rosenberg do not describe using a "body suit" comprising actuators "in the body 
suit". However, a body suit comprising tactile actuators located in the interior of a suit 
was known in the art at the time of the invention. Tremblay teaches a man-machine 
interface to provide tactile feedback to various sensing body parts of the user, wherein a 
body suit having tactile actuators (vibrotractile units "mounted inside or outside of the 
suit in pocket-like cavities" [0066] and Figure 10a) receive tactile signals from 
corresponding tactile sensors on a robot from a network (contact sensors on the robot 
(element 1912), are fed to the computer and sent to vibrotactile units on the user 
[0077]). Even if sensors and actuators are directly attached to users in the teachings of 
Simmons and Rosenberg, as Applicant contends, it would have been obvious to one of 
ordinary skill in the art at the time of the invention to use a body suit, with actuators 
sewn inside or outside the suit, instead of actuators "explicitly affixed to the body" 
(Tremblay, [0066]) because as a Tremblay suggests the two means of transmitting 
haptic sensations to the user are interchangeable: vibrotactile units may be positioned 
affixed to the user's body or if a garment is worn, "then the units may also be mounted 
inside or outside of the suit in pocket-like cavities, and need not be explicitly affixed to 
the body." ([0065], figures 9, 1 0a, 1 0b, 22) The substitution of using a garment with 
actuators and sensors mounted inside or outside for directly placing actuators and 
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sensors on a user would have obtained predictable results of transmitting and receiving 
the same signal information. 

(Claims 2 and 14) Simmons further describes motion sensors positioned 
throughout the body suit (column 13, lines 50-67), the motion sensors sending motion 
signals corresponding to movements of each sensor relative to a reference point 
(column 14, lines 30-41), the motion signals transmitted to the communications network 
(column 7, lines 29-32); and the humanoid robot, receiving, from the communications 
network the signals from the motion sensors (column 1 1 , lines 1 5-60), the signals from 
the motion sensors causing a movement of the robot that is correlated to a movement of 
the body suit (column 8, lines 23-30). 

(Claims 3 and 1 5) Simmons further describes wherein the robot includes 
actuators corresponding to the motion sensors, the actuators causing the robot to move 
(figure 2). 

(Claim 4) Simmons describes the robot comprising a body (column 5, 40-58); a 
camera coupled to the body, the camera for sending video signals to the 
communications network (column 7, lines 2-8, column 1 1 , lines 51 -52); and suggests 
sending audio information to the local site over the communications network (column 
1 1 , lines 51 -52) further describing that a sound sensor means is coupled to second 
body (robot) to capture sound for sending audio signals to the communications network 
(column 4, lines 20-28 and claim 49). 
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Claim Rejections - 35 USC § 103 

Claims 5, 8, 9, 16 and 18 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Simmons in view of Mcintosh, Rosenberg and Tremblay, as applied 
to claims 4 and 14 above, and further in view of Simmons US 20030030397 (Simmons 
'397). 

(Claims 5, 8, and 16) Simmons describes wherein the user wears a "wrap 
around video display or a holographic display over his eyes" to render the video signals 
received from the camera (column 9, lines 5-10) and a transducer to transduce the 
audio signals received from the sound sensor coupled to the robot (claim 49, column 4, 
lines 20-28). Simmons '397 teaches that the robot would comprise microphones in the 
appropriate ear location to be relative to the ear position of the user (claim 19) and that 
the head display would comprise of goggles or glasses (claim 1 2). It would have been 
obvious to one of ordinary skill in the art at the time of the invention to combine the 
teachings of Simmons '397 with the invention of Simmons, in view of Mcintosh and 
Rosenberg, because as Simmons '397 suggests, the microphones and the goggles are 
one well known means of creating the 360-degree, stereoscopic, realistic feedback of 
the remote location ([001 6]). Simmons further describes wherein the virtual reality 
system comprises an interface having one or more channels configured to receive the 
audio signals form the sound sensors (column 7, lines 29-32); receiving the video 
signals form the camera (column 7, lines 2-8); sending the video signals to the head 
display (column 9, lines 5-10); and sending the audio signals to the sound producing 
means (column 4, lines 20-28). 
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(Claims 9 and 1 8) Simmons further describes wherein the body includes an eye 
socket and the camera is positioned in the eye socket (column 7, lines 2-8). 

Claim Rejections - 35 USC § 103 

Claims 10 and 19 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Simmons in view of Mcintosh, Rosenberg, Tremblay and Simmons '397 as applied 
to claims 5 and 16 above, and further in view of Algazi, US 7,333,622. 

Simmons and Simmons '397 describe wherein the sound sensor is positioned on 
the robot relative to the position of the sound receiver on the person and can be refined 
by the shape of the outer ear (column 4, lines 20-28). Algazi teaches it was well known 
in the art to place listening devices in a mannequin having the exact size, shape, and 
acoustic properties of the listener located in the ear canals to replicate the sound 
signals accurately (column 3, lines 22-52). It would have been obvious to one of 
ordinary skill in the art at the time of the invention to combine the teachings of Algazi 
with the invention of Simmons in view of Mcintosh, Rosenberg, Tremblay and Simmons 
'397 because Algazi describes merely one means of achieving the result described by 
Simmons of replicating the 360 degree, precise audio feedback to the remote user. 
Claim Rejections - 35 USC § 103 

Claims 6, 7 and 17 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Simmons in view of Mcintosh, Rosenberg, Tremblay, and Simmons '397 as 
applied to claims 5 and 16 above, and further in view of Abbasi, US 6,786,863. 

Simmons describes a remote robot operating system and method wherein one 
user controls a robot that duplicates the actions of the operator and transmits the 
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sensed condition of the robot environment to the operator, wherein the sensed 
environment is overlaid visually and reproduced by actuators and sensors on the 
operator (column 1 , line 36-column 2, line 12). Simmons does not describe wherein at 
the location of the operator, a second humanoid robot transmits data to a first location. 
However, Abbasi teaches this duplication of the same system to create an interaction 
between remote users is known. Abbasi teaches a remote physical contact system and 
method wherein a first surrogate (robot) is at a first location, a second surrogate (robot) 
is at a second location, the second surrogate having the same components, actuators, 
and sensors, i.e. a second microphone and second camera (figure 1 , elements 35B, 
40B, and 45B); a second display to receive the video signals from a first camera, a 
second earphone to receive audio signals from a first microphone (figure 1 , elements 25 
and figure 6), and a first gateway device in the first location and a second gateway 
device having a processor in the second location to create the remote interaction via a 
network (computer network 30 between computers 15 and 25, figure 1). It would have 
been obvious to one of ordinary skill in the art at the time of the invention to combine the 
teachings of Abbasi with the invention of Simmons, in view of Mcintosh, Rosenberg, 
Tremblay, and Simmons '397 , because as Abbasi teaches the use of remote 
surrogates and expands the notion by using dual surrogates for teleconferencing or 
computer communications, adding a capability to engage in all types of physical contact 
to "provide for the tactile sensation so inherent in many forms of human contact." 
(Column 1 , lines 44-64.) Further the combination of the prior arts would produce a 
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predictable result by merely duplicating the known systems and interchanging the 
physical locations, as clearly suggested by Abbasi. 

Claim Rejections - 35 USC § 103 

Claims 1 1 , 1 2, 20 and 21 are rejected under 35 U.S.C. 1 03(a) as being 
unpatentable over Simmons in view of Mcintosh, Rosenberg, Tremblay, Simmons '397, 
and Algazi as applied to claim 10, Simmons in view of Mcintosh, Rosenberg, and 
Temblay as applied to claim 4, and Simmons in view of Mcintosh, Rosenberg, Tremblay 
and Simmons '397 as applied to claim 16, and in further in view of Yee, US 6,01 6,385. 

Simmons in view of Simmons '397 describes transmitting video signals to a set of 
goggles but does not specify a receiver or that the data is transmitted wirelessly. 
However, Yee teaches a headset of a user comprises a receiver to receive video 
signals (column 5, lines 1 1 -37) and wherein the robot comprises a transmitter to 
wirelessly send the audio, tactile, motion and video signals to the communications 
network (communications antenna 30). It would have been obvious to one of ordinary 
skill in the art at the time of the invention to combine the invention of Simmons in view of 
Mcintosh, Rosenberg, Tremblay and Simmons '397 with the teachings of Yee because 
it was well known that wireless means would offer the predictable result of more 
accessible travel of the robot and a wider range of motion. 

Conclusion 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to CHRISTINE BEHNCKE whose telephone number is 
(571) 272-8103. The examiner can normally be reached on 8:30 am - 5 pm. 
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If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Thomas G. Black can be reached on (571) 272-6956. The fax phone 
number for the organization where this application or proceeding is assigned is 571 - 
273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

CMB 

/Thomas G. Black/ 

Supervisory Patent Examiner, Art Unit 3661 



